Oregon Department of Education 

255 Capitol Street NE 

Salem, Oregon 97310 



Office of Educational Improvement 
and Innovation 
December 2004 




Oregon 

Consolidated State Performance Report 
Comprehensive School Reform Program 



2003-2004 





2004 Oregon Consolidated State Performance Report: Comprehensive School Reform Program (CSR) 



Oregon Consolidated State Performance Report: 

Comprehensive School Reform Program (CSR) 

2003-2004 

The Oregon CSR Consolidated State Performance Report was completed by the 
Oregon Department of Education (ODE) staff. The Northwest Regional Educational 
Laboratory (NWREL) assisted the staff with analysis of implementation trends for Round 
2 CSR schools in their final year of implementation and compiled and analyzed student 
achievement data for all Rounds. This report is submitted in fulfillment of the federal 
ESEA CSR program requirement. 

If you have any questions or need clarification on this report, please contact either of 
these individuals: 

Oregon CSR Coordinator: Janet BubI 503 378-3600 Ext. 2710 

Education Program Specialist janet.bubl@state.or.us 

Director of Support to 503 378-3600 Ext. 4504 

Districts Team: Helen Maguire helen.maguire@state.or.us 

Overview 

Fourteen schools in Oregon have completed their three-year implementation plans as a 
part of Oregon’s Round 2 funding cycle. Seven schools are currently in Round 3 and 
eight schools are currently in Round 4. Oregon SEA staff worked closely with these 
schools as they evaluated progress made during their implementation. An Oregon CSR 
Progress Report Presentations (Appendix A) format was developed and includes the 
following criteria: 

• Progress made on Oregon Statewide Assessments; 

• Progress made on local assessments; 

• Progress made on program implementation of the eleven CSR components. 

This report includes an analysis of Rounds 2, 3 and 4. Round 1 schools had already 
completed their implementation by 2002-2003 and were included in the last state 
evaluation. Table 1 provides Round information by year. From 1999 to 2004, a total of 
52 schools have received CSR funding in Oregon. 

Implementation of Years by Rounds 



Round 


Number 

of 

Schools 


Planning 

Year 


Yearl 

Implementation 


Year 2 

impiementation 


Years 

impiementation 


Round 2 


14 


2000-2001 


2001-2002 


2002-2003 


2003-2004 


Round 3 


7 


2001-2002 


2002-2003 


2003-2004 


2004-2005 


Round 4 


8 


2002-2003 


2003-2004 


2004-2005 





Table 1 
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Process Used to Determine School Selection for All Rounds 

Oregon has a Two-Step Process: Pre-Qualification Application (Step One) and CSR 
Application Information (Step Two) for determining school selection. This has been a 
consistent process since Oregon developed its CSR plan. For purposes of this report, 
the most current documents of the Two-Step Process were included (Appendix B and 
C). While Round 2 schools, specifically, may have had a slightly different form, it 
seemed more appropriate to include up-to-date CSR material. 

• As part of the process, a school district is invited by ODE to apply on behalf of a 
school(s) which meet the CSR criteria. At the time of the grant funding, all CSR 
schools were identified as low-performing schools according to state criteria. 
Currently, only one of Round 2 schools remains in low-performing status. 
However, it should be noted that in 1998-99, when schools were first identified 
for CSR eligibility, the average RIT score on state assessments was the only 
factor used to determine low-performing status. The Oregon Report Card 
evaluation system, which applies to all schools in the state, was implemented in 
January 2000 to measure the performance of Oregon schools. As a result, the 
criteria for identifying low-performing schools has changed to incorporate a 
broader sense of performance indicators. The criteria for Rounds 3 and 4 was 
whether they received Low or Unacceptable on the Oregon Report Card or didn’t 
make Adequate Yearly Progress. 

• The district submits the Pre-Qualification Application (Appendix B ) which is 
scored by an ODE team. A determination is made if the district is ready on 
behalf of the school(s) for the CSR process to begin. Then an on-site school visit 
is made by Oregon Department of Education staff. This is a strength of the 
process as the school staff has time to ask questions when the ODE staff 
presents the CSR process. The ODE staff also has an opportunity to visit the 
school to meet the staff and administration. The staff must have a vote of at 
least 90% agreement to proceed with the application. 

• The school has the remaining school year as a planning year. Title I schools 
were given a $10,000 planning grant from separate funds (non CSR funds) to 
complete the application. Non-Title I schools were given financial assistance 
through their districts. 

• Schools spend the planning year completing the Step Two: CSR Application 
Information (Appendix C). The process includes, for example, a comprehensive 
needs assessment, a prioritization of needs, an inquiry approach to selecting 
research-based program(s), and a detailed implementation action plan. The 
application is completed with support by ODE staff or other technical assistance 
(i.e., school reform consultants, retired educators. Northwest Educational Lab 
staff, district staff). The school is assured that the application will be reviewed 
and evaluated and if there are sections that do not meet the criteria, the school 
will be given additional assistance. The application process is meant to be a 
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positive experience since the school knows that the funding is assured once the 
criteria is met. 

• Schools receive their grant funding and begin their implementation in the fall, 
following the planning year. 

Technical Assistance Provided to Sites 

Technical assistance is provided to all CSR schools in a variety of ways: 

• All participating schools had two workshops per year (2000-2001 , 2001 -2002): 
one in the fall for evaluation training and one in the spring for progress report 
training. In the fall of 2003-2004, the workshop was postponed until January of 
2004. That workshop focused on evaluation analysis and planning for the spring 
Progress Report Presentation at the same time. These were regional workshops 
around the state in four different locations. Although these schools were aware 
of the evaluation component of the CSR program, in the fall this was an 
opportunity to become more familiar with the reporting form and the specific 
criteria for determining progress and continuation of funding. Each school team 
had time to analyze and reflect on data it had gathered to that point and to begin 
work on their reporting documentation. In the spring the work continued with 
emphasis on self-ranking on the CSR eleven components and the upcoming 
Progress Report Presentations. These workshops were also organized by 
Rounds so that schools could be given appropriate support at their level of 
implementation. 

• The workshops were followed by on-going technical assistance by staff from the 
Oregon Department of Education Title I staff and the one remaining ODE Title I 
Distinguished Educator (see next section on Changes in the Oregon CSR 
Process) as time and personnel permitted. In some cases, especially in 2003- 
2004 when the ODE staff was reduced. Northwest Regional Education 
Laboratory evaluation staff. Education Service Districts’ staff, local school district 
staff, school reform consultants, and others such as retired educators who have 
school reform background came to a school’s assistance. The purpose of this 
follow-up assistance was to ensure that each school was able to reflect on the 
progress made during the first year, problem-solve barriers to successful 
implementation, refine goals as needed, and plan for the coming year. 

• The NWREL CSR Unit assisted ODE staff in planning, evaluating and supporting 
CSR implementation. The unit also assisted with all ODE sponsored workshops 
mentioned above. 

• An increase in district level support occurred because of the reduction in the 
ODE staff available for direct school support. ODE staff worked with district 
personnel to help them understand the CSR process and requirements. At each 
workshop, a district person was required to attend. 
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• Direct school support did continue by ODE staff when appropriate or specifically 
needed. CSR principals and school facilitators did receive monthly updates by 
the CSR Coordinator. Support generally included phone conversations, e-mails, 
materials sent, and, in some cases, site visits. 

• CSR has served as a model for an intensive, in-depth school improvement 
planning process. Oregon is electing to use this same model to strengthen the 
Title I Schoolwide Program planning process throughout the state and to provide 
a vehicle for change in schools that are in School Improvement status. All CSR 
schools, schools engaged in Title I Schoolwide Program planning, and schools in 
School Improvement status work with a school support team from various places 
(i.e., ODE staff. Education Services Districts, their own district personnel support, 
school reform consultants, retired educators) . This consistent process is proving 
to be effective and will be strengthened and continued by the various ODE teams 
working together. 

• Schools in all Rounds had technical assistance from Locally Developed 
Programs or Externally Developed Programs. The Locally Developed Programs 
included outside professional development training which was researched by the 
CSR Team during planning. While many stayed with a Model Provider, others 
sought specific technical assistance for their respective school. (For the purposes 
of this report either designation of Externally Developed Program or Model 
Provider will be used). 

• Up until 2003-2004, model providers and ODE and/or district staff was generally 
used by CSR schools. But with the reduction of Distinguished Educators and an 
awareness that the model providers were not giving schools the in-depth 
technical assistance on the entire CSR eleven components and school reform, a 
requirement that each school contract with a Technical Assistance External 
Provider (also referred to as just an External Provider) for 2004-2005 was 
established. The document schools had to complete, 2004-2005 Requirement 
for External Support and Technical Assistance for CSR Schools and Districts is 
included (Appendix D). Several CSR schools had been contracting with an 
independent, specific Technical Assistance External Provider during 2003-2004. 
She was not associated with a model provider but had extensive school reform 
experience. It became apparent to the ODE CSR Coordinator that an 
independent External Provider would be important for all CSR schools. The 
particular External Provider has become this year’s trainer/support for all the new 
External Providers this year. She contacts all the providers through e-mail and/or 
the phone, sends information about CSR and technical assistance, and will host 
a workshop in January 2005 for all the external providers. 

• This group of Technical Assistance External Providers throughout the state are 
from various backgrounds: school reform consultants, retired educators, retired 
superintendents, and Education Service District support, for instance. By 
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December 2004, 10 external providers were working with 15 Round 3 and 4 
schools; two providers were working with more than one school. 



In summary, the strength of the Oregon CSR process is its belief that comprehensive 
reform is complex and unique to each school and district. For this reason, a “one-size 
fits all” approach would not capture the effort and progress each school was 
accomplishing. Although the criteria are the same for each CSR school, Oregon’s 
approach provides an individualized assessment of each school’s progress. This 
approach considers the context in which the school is operating when determining 
whether progress has been made. By individualizing the process, Oregon expects each 
school to be different, although each is expected to show progress. As a result, the 
technical assistance offered is tailored to meet the different needs of each school. It is 
because of the individual focus on each school that changes in the technical assistance 
was made for 2004-2005. 



Changes in the Oregon CSR Program 

Certain change have occurred at the Department of Education; 

• The ODE Distinguished Educator program, which had eight support staff 
working with schools throughout the state, was reduced to one Distinguished 
Educator in 2002-2003. Currently, her responsibilities also include Title lA 
Schoolwide Planning and Targeted Assistance support. School Improvement 
Grants and various other Title lA responsibilities. 

• Another major changed occurred in the CSR progress report process. Originally, 
progress reports were written each spring and submitted to ODE. These were 
reviewed and a written report was sent to the school in the fall. Beginning in 
2002-2003 and 2003-2004, schools began giving Progress Report Presentations. 
This will be fully explain in the following section: Process and Criteria Used to 
Determine Sites Made Progress. 

• Another change for Oregon, was the retirement of Chris Rhines who established 
and coordinated the Oregon CSR (originally the CSRD) since its beginning. 

Janet BubI, ODE Education Program Specialist, was named the CSR 
Coordinator in October 2004. 



Oregon School Information 

The following section describes CSR schools in various ways: enrollment, poverty, 
student characteristics, state designations, grant funding, etc. 

• Table 2 provides information about Oregon school enrollment and the average 
school size. CSR schools are generally small, with a few exceptions. While a 
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small school may be helpful in staff communication, often resources, in general, 
are limited. 



CSR Enrollment 
2003-2004 



Round 


Under 300 


300-600 


600-900 


900 or more 


Average 


Enrollment 




Enrollment 


Enrollment 


Enrollment 


Enrollment 


Enrollment 


Range 


2 


6 


4 


3 


1 


429 


186 to 1108 


3 


3 


4 


0 


0 


328 


127 to 467 


4 


3 


3 


2 


0 


350 


72 to 777 



Table 2 



• Table 3 provides information about the breakdown of schools in the CSR 
program. Perhaps the most interesting is the number of alternative schools. 

One alternative schools is from an urban setting and includes both middle and 
high school students. The other alternative school is in a rural setting and only 
serves high school students. Both are in Round 4, which is also interesting in 
comparing the success they are having. Obviously, their work has only begun as 
they are in their second year of implementation. It so happens that both schools’ 
External Provider has been the same person for two years. She has encouraged 
their networking of information and a site visit by the schools. 

• It is also apparent, as mentioned earlier, CSR schools changed from 
predominately Externally Developed (model) Programs to Locally Developed 
Programs. Many schools in the various Rounds retained their Externally 
Developed Programs (Table 3). Out of 23 school programs in Oregon, 20 are 
using Externally Developed Programs and 9 are using Locally Developed 
Programs. The Locally Developed Programs have occurred mostly in Round 4. 
This was probably due to the clarification in the CSR legislation that schools did 
not have to choose an Externally Developed Program. 



CSR Schools and Developed Programs 
Through 2003-2004 



Round 


Number of 
Elementary 
Schools 


Number of 
Middle 
Schools 


Number 

of 

High 

Schools 


Number of 
Alternative 
Schools (also 
included in the 
previous 
columns) 


Number 

of 

Externally 

Developed 

Programs 


Number of 
Locally 
Developed 
Programs 


2 


3 


7 


4 


0 


12 


2 


3 


3 


3 


1 


0 


6 


1 


4 


3 


2 


3 


2* 


2 


6 



Table 3 

*One alternative school is only a high school and one alternative school is middle and high scheol combined. 
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• Table 4 provides the number of urban and rural schools. Oregon has three major 
urban centers (Portland, Salem and Eugene) which all happen to be in the 
Willamette Valley. A number of schools are in rural and even remote areas. 
Southern Oregon and Central Oregon are some distance from the Willamette 
Valley, some separated by the Cascade Mountain Range. This has been a 
problem for workshops, site visits, and finding and using Technical Assistance 
External Providers or Model Providers. The distance has meant that, at times, 
ODE staff will travel to the sites for smaller workshops (combining several rural 
districts) rather than having the schools travel to Salem, the location of the 
Oregon Department of Education. Occasionally, ODE will host a workshop in the 
Portland area which includes a number of other cities in the Portland 
Metro/Multnomah County region. When noting the number of visits by Model 
Providers or other outside consultants (presented as a document in schools’ 

CSR Progress Reports), it is apparent that the rural and even more remote sites 
are at a disadvantage. 

• Table 4 also shows the range of poverty in the schools and the number of Title I 
schools. In reviewing demographics for the various Rounds, it is apparent that 
poverty numbers are significant in Oregon. Currently, Oregon is ranked second 
in the nation for hunger; last year Oregon was ranked first. Oregon has the 
highest unemployment in the nation. In 2002, Oregon was ranked 29 out of 50 
states because 18% of Youth and Children Under Age 18 were living in poverty. 
Most schools, as noted, are Title I schools. Being a Title I school has been an 
advantage to CSR schools in that they have had experience with managing 
funds, working on professional development, having a sense (though often 
without an in-depth needs assessment) of schoolwide planning and creating an 
implementation action plan. CSR planning increased their use of a 
comprehensive needs assessment and a more in-depth action plan. 



Rural, Urban, Title I Schools and Poverty Figures 

2003-2004 



Round 


Number of Urban 
Schools 


Number of Rural 
Schools 


Number of Title 1 
Schools 


Range of Poverty in 
Schoois 


2 


7 


7 


10 


37.5% to 82% 


3 


2 


5 


7 


56% to 74% 


4 


4 


4 


7 


54% to 88% 



Table 4 



• Table 5 provides various data points about Students with Disabilities and English 
Language Learners (ELL). In urban schools in Oregon, the number with multiple 
languages spoken in the home other than English is significant. Some urban 
schools have as many as 1 7 to 23 languages spoken in the homes. During CSR 
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Progress Reports, schools self-reported that communication with parents, whose 
native language was not English, was very difficult. This was a constant 
challenge and theme for schools. The numbers of ELL students is also a factor 
in Oregon schools. For instance, one elementary school reported its enrollment 
of ELL students is 73% (it also happens to be the elementary school with the 
highest poverty at 88%). The ELL figures show that Oregon’s population is in 
need of ELL support at the school level. The overall average percentages for all 
CSR schools are above the state ELL Oregon student average in 2004 of 9.4%. 
The range is important as well. Individual schools have some very large numbers 
of ELL populations, up to 85%. Students with Disabilities are within a normal 
range of figures one would expect in schools and compare to the state student 
average in 2004 of 1 2.9%. 



Students with Disabilities and English Language Learners 

in CSR Schools 
2003-2004 



Round 


Average % of 
Students with 
Disabilities 


Range of 
Students with 
Disabilities 


Average % of 
English 
Language 
Learners 


Range of 
English 
Language 
Learners 


2 


7% 


0%to 16% 


13.8% 


0% to 45.9% 


3 


6.4% 


1%to 10% 


10.7%' 


0% to 1 9.3% 


4 


7.5% 


3% to 14% 


19.5% 


0% to 85% 



Table 5 



• Table 6 provides Oregon Report Card and Adequate Yearly Progress (AYP) 
status. It is hopeful to see that 1 00% of Round 3 schools are rated “Satisfactory 
or Strong” and 86% or them have met AYP. [Schools are analyzed by four 
categories on the Oregon Report Card: Student Performance, academic 
achievement; Student Behavior, attendance/dropout; Improvement; and Student 
Characteristics, participation in statewide assessments. Schools are rated 
Exceptional, Strong, Satisfactory, Low or Unacceptable. A school’s “meeting” 
AYP is determined by students, including subgroups, needing to meet rigorous 
state academic standards by 2013-2014, thus needing to meet annual 
performance targets.] As might be expected, because Round 4 schools were 
finishing the implementation on only their first year at the time of the AYP data 
was represented, they have some work to do. One would have hoped that 
Round 2 schools would have done as well in AYP as Round 3 schools, but that 
doesn’t appear to be the case. 
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Oregon Report Card Rating and AYP Designation 
for CSR Schoois 
2003-2004 



Round 


Percentage of Schools with 
a Rating of Satisfactory or 
Strong on the Oregon 
Report Card 


Percentage of Schoois that 
Met Adequate Yeariy 
Progress 


2 


78% 


42% 


3 


1 00% 


86% 


4 


63% 


50% 



Table 6 



• Since one of the factors in student success is attendance, Table 7 provides 
attendance data compared to state data. All CSR elementary schools had an 
attendance level below the state. CSR middle schools also fell below the state 
average. The most significant difference is in the high schools which fell well 
below the state average. There are two alternative schools which likely affected 
the average for the high schools; however, one alternative school’s attendance 
average is 90%, but the other’s is 82.9%. While this may account for a poor 
average, the issue of high school attendance is still a concern for schools. 



CSR Attendance Averages Compared to State Averages 

2003-2004 





CSR 


State 


CSR 


State 


CSR 


State 




Eiementary 


Eiementary 


Middie 


Middie 


High 


High 


Round 


Average 


Average 


Schooi 


Schooi 


Schooi 


Schooi 




Attendance 


Attendance 


Average 


Average 


Average 


Average 








Attendance 


Attendance 


Attendance 


Attendance 


2 


93.5% 


94.2% 


92.8% 


94.2% 


88.3% 


91.4% 


3 


93.9% 


94.2% 


93.6% 


94.2% 


87.6% 


91.4% 


4 


93.6% 


94.2% 


92.3% 


94.2% 


86.4% 


91.4% 



Table 7 



• Without exception, each Round has had a minimum of two changes in principals 
during the CSR grant implementation. One school had a change each year, 
beginning at the end of the planning year and each year thereafter. What was 
most apparent by the change in school administration was the lack of knowledge 
and the time it took, especially in the fall, to involve the principal in the CSR grant 
process. If a school had a strong CSR team and/or planning team, then the 
school tended to stay more on track. If a school had a change in the team or if 
the team, itself, was not directly involved (in cases where the principal held most 
of the information), the school and the new principal were at a serious 
disadvantage. These events also occurred at the same time that the 
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Distinguished Educator program was significantly reduced so there was not ODE 
staff available to go to each school in September to help the principal get 
engaged in the process. 

• Even at the district level there have been administrative changes, both in Title I 
coordinators and district superintendents. This has meant that schools who were 
given permission to proceed with the CSR application, including flexibility 
promised by the district, often found the district unaware of earlier agreements. 
Educating those in the district office was time consuming. Often ODE staff found 
it necessary to meet with the new coordinators and superintendents in order to 
explain what the District had “agreed to” when it originally applied on behalf of 
the school(s). 

• Table 8 gives the average CSR grant amount per school, including the district 
administration amount. At first, funds were distributed equally per school. 
Subsequently, schools were awarded various amounts depending on the school 
size/per child allocation. A base amount was determined for the year and the 
Round, then additional funds were awarded. The reason the amount was 
increased from $76,650 was that it did not seem to be adequate to fund school 
reform and a fixed amount per school was also an issue because of school size. 

• It is noted that there was a significant increase in grant amounts in the past two 
years. With three CSR schools withdrawing, the funds were reallocated to the 
other schools. In all three cases, the decision to withdraw was the school’s (by a 
staff vote) and not a state’s decision. It appeared that two main reasons for the 
withdrawals were because the lack of complete teacher buy-in and a change in 
the principal. For two schools, even though there was buy-in during the initial 
application and implementation process, the buy-in changed over time. The third 
school went through the planning year and with a new out-of-state principal who 
was not in favor or interested in the CSR program, that school did not even start 
its implementation. 



CSR Grant Amounts 



Round 


Average Grant 
Amount 
Yean 

Implementation 


Average Grant 
Amount 
Year 2 

Implementation 


Average Grant 
Amount 
Years 

Implementation 


Funding Range 
During Implementation 
Years 


2 


$76,650 

All schools received the 
same amount that year 


$87,868 


$95,576 


Low of $73,000 to 
high of $114,260 


3 


$83,515 


$91,791 


$168,098 

(This year, 2004- 
2005) 


Low of $77,250 to 
high of $173,449 


4 


$93,923 


$169,898 

(This year, 2004- 
2005) 




Low of $83,600 to 
high of $188,893 



Table 8 
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• Table 9 presents the increase in grant funds over the three years. It is clear 
that early grants were smaller than later years. The change, as previously 
mentioned, occurred because schools needed more funding for reform. The 
size of schools also become a relevant issue in funding. As a note, in 
reviewing Round 2 schools, the grant funding can be broken down into school 
levels and averages: the elementary school average over three years was 
$84,030; the middle school average over three years was $86,352; and the 
high school average over three years was $89,841 . 



Range of Grant Funding from 
2002-2003 to 2004-2005 



Year of 
Grant 
Funding 


$70,000 to 
$90,000 


$91 ,000 to 
$130,000 


$131,000 to 
$190,000 


Total Number of 
Schools in 
Implementation 


2002-2003 


24 


5 


0 


29 


2003-2004 


12 


9 


8 


29 


2004-2005 


5 


9 


7 


21 



Table 9 



• Supplemental funding (not reflected in Table 8) has been offered to CSR 
schools for the past two years for summer work. Schools could apply for a 
maximum of $16,000. Schools applied for the money by stating a plan based 
on need, determining a summer “schedule,” and developing a budget. Some 
schools used the money for a student summer school and others used the 
money for staff development. All but one school, from all Rounds, applied 
and received supplemental funding. 



Process and Criteria Used to Determine Sites Made Progress 

• Beginning in May/June 2003, and continuing May/June 2004, school teams were 
required to give an oral CSR Progress Report Presentations (Appendix A). 

School teams prepared an hour report, followed by 30 to 40 minutes of questions 
by the State Review Team (SRT). The size of the school team varied from 
usually 4 but occasionally up to 8 to 1 0 staff. Schools teams were required to 
bring the principal and a district person to the Oregon Department of Education in 
Salem, where the reports were given. Some schools included External Providers 
(which will be required in the next progress report process in June 2005). 
Generally, schools gave a powerpoint presentation with each staff member giving 
part of the presentation. Previously, schools wrote and submitted a progress 
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report. Each team was asked to compare the process and value of the written 
report versus the oral presentation. Every school, without question, said the oral 
presentation was just as time consuming but a much more inclusive process. 

Staff on the team met repeatedly to discuss, plan and prepare the report. Rather 
than just a few (or even one person) writing the report it was a true group effort 
and a considerable amount of school reflection occurred. Often it was noted that 
schools presented their report to the entire staff for input before the school’s 
presentation to ODE. Some used parts of the report to present to school boards 
and parent groups and/or site councils. This appeared to give buy-in to all 
stakeholders at the site, schools reported. Schools had specific information and 
documents to be presented and included in hard copy. The CSR Progress 
Report Presentations (Appendix A) was the school’s guidance. 

• The SRT consisted of Oregon Department of Education staff, the ODE Title I 
Distinguished Educator, Northwest Regional Education Laboratory evaluation 
staff. Education Service Districts’ staff, local school district staff, and others with 
school reform background. Principals from Round 2 CSR schools were also 
asked to be on the SRT. The teams were varied in size, depending on the 
schedule, but generally there were at least 4 to 5 team members at each 
presentation. Two people, the CSR State Coordinator and one other person very 
knowledgeable in the CSR process and school reform, listened to all 29 reports. 
The SRT used the same CSR Progress Report iorm as it listened to the 
presentations and made notes for a final written report. Comments from 
individual SRT members’ forms were consolidated into one report and final 
determination of progress was made. Each school and district received a School 
and District Progress Report with a summary of its presentation with findings by 
the SRT which included positive comments, feedback, clarifying questions, 
strengths and areas of concern, and other comments to consider as the school 
either continued in implementation or if it is was in its last year, sustainability 
considerations. A sample from one school’s final State Review Team’s 
comments, the School and District Progress Report Review Team Comments, is 
included (Appendix E). 

• Each school, in all rounds, was required to determine if at least some progress 
was made toward student achievement goals, either through Oregon Statewide 
Assessments or local assessment (see School Progress in Implementation 
below). Schools were also required to self-evaluate the progress made toward 
successful implementation of the eleven components using the Continuum for 
Assessing the Comprehensive of School Reform Plans (Appendix T) and to 
submit an action plan and budget for the following year, besides a number of 
other documents. Schools were required to report progress on either state 
assessment or local assessment data. An analysis of these data determined 
whether a greater percentage of students met or exceeded state standards in 
reading and math, and whether gains were made on local assessment data. 
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• In the fall of 2004, as Rounds 3 and 4 schools began their next school year, the 
External Provider for each school was asked, specifically, to review with the 
school team (who gave the presentation) and/or CSR Team the State Review 
Team’s written report on the school’s Progress Report Presentation. This gave 
the schools an opportunity to review the report and to begin addressing concerns 
or questions the SRT had. It also gave focus to the CSR plan at the beginning of 
the year. In the fall of 2003, this was not as easily handled as not all schools had 
an External Provider who could review with them. Since the requirement has 
changed (all schools contracting with an External Provider), this fall it happened. 
In June Round 2 schools were asked specific questions about how they 
proposed to sustain their plan at the end of their funding. Since Round 2 
schools did not have external provider support any more, the local school district 
was expected to work with the school in reviewing the SRT report in the fall of 
2004. 



Student Performance Data: State Assessments 

• The Oregon State Assessments (OSAT) are given yearly in Reading and Math at 
grades 3, 5, 8, 10. Both are multiple choice content assessments. In addition, 
but not included in the CSR report, are two other assessments: Math Problem 
Solving and Writing. These are sent to the state and scored by teachers 
throughout the state with an appropriate scoring guide. In the past, students 
have used paper and pencil, but currently many students use the Technology 
Enhanced State Assessment and take the multiple choice tests on a computer. 

• Table 1 0 provides the number of schools that showed Oregon State Assessment 
gains of students meeting or exceeding standards from 2001-2004 (Round 2) or 
those schools not showing gains. State test scores for Round 2 schools were 
examined over the four years (initial planning year and three implementation 
years) to identify the percentage change. [The Round 3 schools show data from 
2002-2004 and Round 4 shows data from 2002-2004 in following Tables.] 

• District data are not compared because schools are often the only school in the 
district in their grade level. 
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Round 2 Schools Compared to State Data 
(Planning Year 2000-2001, Implementation Years 2001-2002 to 2003-2004) 



Subject/Grade 


Total number 
of Round 2 
schools testing 
In this grade 


Number of 
Round 2 
schools 
showing gains 
2001-2004 


Percent of 
Round 2 
schools 
showing gains 
2001-2004 


Round 2 
schools that did 
better than the 
statewide 
change from 
2001-2004 


Percent of 
Round 2 
schools that did 
better than the 
statewide 
change 














Reading 












Grade 3 


2 


0 


0% 


1 


50% 


Grade 5 


6 


3 


50% 


4 


75% 


Grade 8 


7 


2 


29% 


2 


29% 


Grade 10 


4 


3 


75% 


3 


75% 


All Grades 


19 


8 


42% 


10 


53% 














Math 












Grade 3 


2 


2 


100% 


2 


100% 


Grade 5 


6 


5 


83% 


5 


83% 


Grade 8 


7 


4 


57% 


3 


57% 


Grade 10 


4 


0 


0% 


0 


25% 


All Grades 


19 


11 


59% 


10 


53% 



Table 10 



The data in Table 10 show that the 14 Round 2 schools were slightly more likely to 
show gains than not over the grant period. Three key points can be drawn from the 
data in the above table: 

• From 2001 to 2004, 8 of the 19 tested grades in the CSR schools showed gains 
in reading and 1 1 showed no gains. In math, 11 of the 19 tested grades in CSR 
schools showed gains. (Note: There are 14 schools in Round 2, but because 
some schools have more than one tested grade, there are 19 tested grade data 
points for each subject each year.) 

. Comparing the CSR schools to the state in reading, 53% of Round 2 schools (1 0 
of 1 9) did better than the state. Statewide, the percentage of students meeting 
state reading standards in the four tested grades actually dropped slightly from 
2001 to 2004. Two of the Round 2 schools stayed the same or showed a decline 
but still did better than the statewide change. 

. Comparing the CSR schools to the state in math, 53% of Round 2 schools did 
better than the state. The percentage of students meeting state math standards 
in three tested grades (3, 5, and 8) increased statewide from 2001 to 2004 and in 
grade 1 0 the statewide percentage dropped slightly. 
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Round 3 Schools Compared to State Data 

(Planning Year 2001-2002, Implementation Years 2002-2003 to 2003-2004) 



Subject/Grade 


Total number 
of Round 3 
schools testing 
in this grade 


Number of 
Round 3 
schoois 
showing gains 
2002-2004 


Percent of 
Round 3 
schoois 
showing gains 
2002-2004 


Round 3 
schools that did 
better than the 
statewide 
change from 
2002-2004 


Percent of 
Round 3 
schools that did 
better than the 
statewide 
change 














Reading 












Grade 3 


3 


0 


0% 


1 


33% 


Grade 5 


3 


1 


33% 


2 


66% 


Grade 8 


3 


0 


0% 


2 


66% 


Grade 10 


1 


1 


100% 


1 


100% 


All Grades 


10 


2 


20% 


6 


60% 














Math 












Grade 3 


3 


0 


0% 


0 


0% 


Grade 5 


3 


2 


66% 


2 


66% 


Grade 8 


3 


2 


66% 


2 


66% 


Grade 10 


1 


1 


100% 


1 


100% 


All Grades 


10 


5 


50% 


5 


50% 



Table 1 1 



The same mixed results hold so far in Round 3 schools where six of the 10 tested 
grades have done better than the statewide change in reading and five have done 
better than the statewide change in math (Table 11). 

Round 4 Schools Compared to State Data 

(Planning Year 2002-2003, Implementation Year 2003-2004) 



Subject/Grade 


Total number 
of Round 4 
schools testing 
in this grade 


Number of 
Round 4 
schools 
showing gains 
2003-2004 


Percent of 
Round 4 
schools 
showing gains 
2003-2004 


Round 4 
schools that did 
better than the 
statewide 
change from 
2003-2004 


Percent of 
Round 4 
schools that did 
better than the 
statewide 
change 














Reading 












Grade 3 


3 


2 


66% 


2 


66% 


Grade 5 


4 


2 


50% 


2 


50% 


Grade 8 


4 


2 


50% 


2 


50% 


Grade 10 


3 


1 


33% 


2 


66% 


All Grades 


14 


7 


50% 


8 


57% 














Math 












Grade 3 


3 


2 


66% 


1 


33% 


Grade 5 


4 


1 


25% 


1 


25% 


Grade 8 


4 


2 


50% 


2 


50% 


Grade 10 


3 


0 


0% 


1 


33% 


All Grades 


14 


5 


36% 


5 


36% 



Table 12 
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Reading results so far in Round 4 are again mixed: 8 of the 14 tested grades in the 
CSR schools have done better than the statewide change. In math, though, only 5 of 
the 14 tested grades in the CSR schools have done better than the statewide average 
change with just two years of data (Table 1 2). 



Student Performance Data: Number of Students Tested and 
Percentage of Students Meeting Standards 

The tables in the previous section focused on whether or not CSR schools showed 
improvement or declines. The two following tables focus on whether students in CSR 
schools improved or not as a group. In general, the data in these two tables also do not 
show a consistent pattern of improvement for students in the tested grades in the CSR 
Round 2 schools. In both reading and math, for two of the four tested grades, students 
in CSR Round 2 schools did better than the statewide average change and in two they 
did worse (based on comparison of the two rightmost columns in Tables 13 and 14). 



Number of Students Tested and 
Meeting Standards Over Time 

Reading 



Grade 


Number 

of 

Sites in 
Round 2 


Number 

Tested 

Spring 

2001 


Number 

Tested 

Spring 

2004 


%of 

Students 

Meetings 

Standards 

Spring 

2001 


%of 

Students 

Meeting 

Standards 

Spring 

2004 


Round 2 
Change 
2001 
to 

2004 


State 

Change 

2001 

to 

2004 


3 


2 


115 


101 


73 % 


66 % 


- 7 % 


- 2 % 


5 


6 


348 


304 


65 % 


70 % 


5 % 


- 1 % 


8 


7 


861 


934 


51 % 


47 % 


- 4 % 


- 3 % 


10 


4 


446 


422 


37 % 


46 % 


10 % 


- 2 % 



Table 13 



Math 



Grade 


Number 

of 

Sites in 
Round 2 


Number 

Tested 

Spring 

2001 


Number 

Tested 

Spring 

2004 


%of 

Students 

Meetings 

Standards 

Spring 

2001 


%of 

Students 

Meeting 

Standards 

Spring 

2004 


Round 2 
Change 
2001 
to 

2004 


State 

Change 

2001 

to 

2004 


3 


2 


132 


102 


48 % 


71 % 


22 % 


7 % 


5 


6 


361 


311 


64 % 


78 % 


14 % 


3 % 


8 


7 


878 


941 


41 % 


44 % 


3 % 


4 % 


10 


4 


457 


430 


30 % 


22 % 


- 8 % 


1 % 



Table 14 
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While Grades 3 and 5 showed large gains in math, it’s important to note that there was 
a substantial drop in the number of students tested in both grades which indicates there 
probably was a change in the schools’ service boundary and/or student demographics. 
Either factor makes it hard to correlate the gains to CSR. 



Student Performance Data: Local Interim Assessments 

• Each school was required to develop a Language Arts and Math plan to assess 
students with local interim measures at least twice a year (fall and spring). A wide 
variety of interim assessments were used (i.e., pre/post tests in math, Individual 
Reading Inventory, STAR Reading, STAR Math, math problem solving work 
samples, and many others). The average number of assessments given per 
school included assessments at each grade level. The schools reported these 
results by showing the increase or decrease in student achievement, determining 
a school goal for the spring test, and identifying whether or not the school met its 
goal. The schools presented this information during their Progress Report 
Presentations. (See sample Interim Assessment Chart in Progress Report 
Presentations, page 25 of Appendix A.) 

• Tables 1 5 and 1 6 show very few schools meeting their goals in Language Arts 
and Math. This appears to be inconsistent with the high percentage of schools 
which reported students increasing their interim assessment from the fall to 
spring. One issue that impacts schools is the pressure they feel from districts 
and/or NCLB requirements that students are progressing. The column on The 
Number of Schools Where All Student Improved on the Interim Assessment, 
Tables 15 and 16, is a good example. Schools were consistently showing that 
students were making progress, but because they set such high goals, they 
could not rate themselves as meeting their goals (see column The Number of 
Schools that Met at least 75% of Their Self-Determined Goals). Another issue 
may be that schools had difficulty determining an appropriate goal to set. When 
a school, for example, showed a 48% increase in student data, but didn’t reach 
its goal, it appeared that the school had such low fall scores that the school set 
the spring goal very high. While they didn’t meet their goals, students in very 
high numbers were increasing their scores on the spring’s interim assessment, 
whatever assessment it was. 

• A concern about interim assessments appears in the self-reporting part of this 
consolidated report. Schools had a difficult time finding assessments that 
worked effectively for them, especially in math. They also found the interim 
assessments often didn’t align with the Oregon State Assessment Benchmarks. 
Staffs often felt they were not trained to give the tests nor did they have time to 
administer the test. Issues that are very apparent, especially in newly funded 
CSR schools, are administering the assessment, compiling and analyzing the 
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data and using the data to inform instruction and schoolwide goals. It wasn’t 
uncommon for Round 4 schools to be creating the interim assessment 
documentation in the spring just before the next set of tests were to be given. 

• Since interim assessment data was to drive instruction, this apparently didn’t 
happen as frequently as was hoped, due to the complexity of giving assessments 
and compiling information. 

• Future concerns could include: how to help schools determine good 
assessments; how to train staff effectively (especially new staff); how to review 
data and set reasonable goals; how to drive instruction based on data gathered. 



Language Arts 



Local Interim Assessment Results 
2003-2004 



Round 


Number of 
Schools 
Reporting 


Average Number of 
Language Arts 
Interim 

Assessments 
Given per School 


The Number of Schools 
Where aM Students 
Improved on the 
Language Arts Interim 
Assessments 


Number of 
Schools that met at 
least 75% of Their 
Self-Determined 
Goals 


2 


12 


8 


6 


1 


3 


7 


8 


4 


1 


4 


7 


6 


6 


2 



Table 15 



Math 



Round 


Number of 
Schools 
Reporting 


Average Number of 
Math Interim 
Assessments 
Given per School 


The Number of schools 
Where aM Students 
Improved on the Math 
Interim Assessments 


Number of 
Schools that met at 
least 75% of their 
Self-Determined 
Goals 


2 


12 


6 


4 


2 


3 


7 


9 


5 


1 


4 


7 


8 


3 


1 



Table 16 



Round 2: School Progress in Implementation of the Eleven Components 

• Prior to the May/June 2004 Progress Report presentation, the team, site council, 
and staff used the Oregon CSR Progress Report Presentation form and the 
Continuum for Assessing the Comprehensiveness of Schooi Reform Plans 
(Appendix F), from the US Department of Education with minor changes made 
for clarification, to assess their own progress. Schools self-reported progress 
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made toward implementation of each the 1 1 components; however, only 6 of the 
1 1 components were part of the Continuum for Assessing the 
Comprehensiveness of School Reform Plans and were, thus, self-ranked. For 
instance. External Assistance, Support to School were not specifically listed in 
the Continuum but schools did report on them. It became apparent, in analyzing 
data for this report, that in order to reflect on implementation completely in the 
future that schools will need to rank themselves on all eleven components. 
Redesigning the Continuum for Assessing the Comprehensiveness of School 
Reform Plans will make this possible. 

• In the six components, schools identified indicators and ranked themselves in 
one of the three categories: Most Comprehensive (descriptions that best 
represent the spirit of the CSR Program); Somewhat Comprehensive 
(descriptions that should be strengthened to better address the requirements of 
the legislation); Not Comprehensive (descriptions that are inadequate to meet the 
requirements of the CSR program). 

• A federal report review team of ODE staff. Northwest Regional Lab staff, and an 
External Provider to CSR schools met and reviewed all of the School and District 
Progress Report Review Team Comments from all Round 2 presentations. 

These were recorded, tabulated, and synthesized on the document entitled 
Round 2: Analysis and Summary of School Progress on Implementation of CSR 
Plan from Progress Reports May/June 2004 (Appendix GJ. Included in the same 
document is a Chart of Documentation and Progress Report Notes. This 
identified where the information was found by the federal report review team. As 
noted on the chart, some information was directly taken from documentation 
required by the school and some was taken from the SRT written reports. 

The following is the Round 2 summary of the complete analysis of the eleven 
components in the Round 2: Analysis and Summary of School Progress on 
Implementation of CSR Plan from Progress Reports May/June 2004 (Appendix GJ; 

Component #1 : Research-Based Methods 

Out of 14 schools, half ranked themselves “Somewhat Comprehensive” and the other 
half ranked themselves “Most Comprehensive.” Schools found themselves 
implementing the program (either an Externally Developed Program or a Locally 
Developed Program) mainly because of on-site evaluation (by the developer) and 
support to staff. Some schools found that the lack of model developers’ evaluation 
guidelines and a lack of understanding of the local context were factors that hindered 
implementation. Locally Developed Program evaluation was often very helpful. 

One of the problems occurred when a school dropped a Model Provider, then found it 
did not do enough research to develop its own program and strategies. This has been a 
consistent problem because doing research requires time, commitment, and analysis 
skills by staff who are already busy and oftentimes just want to “get on with the grant.” 
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It should be noted that in Round 2, 1 1 schools out of the 14 maintained their model 
provider for the life of the grant. One particular model provider was dropped by two 
schools: one in Round 2 and the other school in Round 4. It is interesting to note that 
the Round 2 school that dropped that particular provider in its last year of 
implementation also received a new principal and even a Title I Coordinator and 
Superintendent for the district changed as well. It wasn’t until the spring, when the 
school was working on the Progress Report Presentation, that the principal began to 
understand the depth of the problem. He did contract with an External Provider who 
supported him and the team as they worked on their presentation. One footnote, is that 
even after the CSR funds ended, that same principal contracted in the fall of 2004 with 
the same External Provider to continue to help the school get “back on track” and try to 
maintain/sustain the program that he felt was finally getting into place. The complexity 
of using a model provider is apparent, especially when it is terminated by the school. 

It should be noted that in Round 3 and 4, eight schools out of the 15 have maintained 
their Model Provider through 2003-2004 (Table 3). The one Round 4 school that 
dropped the model provider lost considerable time because the school year was just 
beginning and CSR was just starting to be implemented. The school had to rewrite its 
plan and determine its own Locally Developed Program. 

Component #2: Comprehensive Design 

Eleven of the 14 schools ranked themselves “Most Comprehensive.” Teams believed 
they integrated subjects, aligned state standards, used a variety of strategies and 
focused on subgroups. Schools also involved all the staff in the program, evaluation 
and design. Those three schools who ranked themselves “Somewhat Comprehensive” 
came to realize that the model, itself, is not the whole program. Continual staff and 
administration turnover also created a loss of buy-in and continuity. Those schools also 
believed that it was easier to be comprehensive in curriculum than comprehensive in 
other areas such as governance and parent and community involvement. 

Component #3: Professional Development 

Again, eleven of the 14 schools ranked themselves “Most Comprehensive.” This may 
be the strongest component for CSR schools. They seemed to use funds to have 
continual, on-site, sustained professional development. Coaches were used to help 
sustain the skilled learned during professional development. Schools who are truly 
engaged in professional development have a teacher mentoring program, have early 
release days, use assessment data to determine professional development, use funds 
to pay for extended hours (for teachers and paraprofessionals). 

The three schools who ranked themselves “Somewhat Comprehensive” listed the 
following factors that hindered implementation: low quality professional development, 
lack of relevance, and paraprofessionals not included in training. With schools that 
dropped their Model Provider, comments about professional development surfaced: 
problems with contractual issues with providers; the lack of congruence between 
professional development provider; and the school’s assessment of their professional 
development needs. 
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Component #4: Measurable Goals and Benchmarks 

In this section, two areas of self-reporting were combined: Measurable Goals and 
Benchmarks and Curriculum Alignment. 

Eleven schools ranked themselves “Most Comprehensive” in Measurable Goals and 
Benchmarks. Training provided about goal writing appeared to be effective. Goals are 
now linked directly to state standards and there is continual analysis of data to reset and 
revise goals. Of the three who ranked themselves “Somewhat Comprehensive,” they 
felt there was not enough involvement in reviewing and revising goals, especially with 
staff, parents, and the community. 

Interim assessment issues included the need for staff training to administer the 
assessments, the lack of alignment between the assessments and state benchmarks, 
and time to administer the assessments were all factors that hindered the 
implementation of this component. 

A more difficult area for schools, even after many years of a published Oregon State 
Standards and Benchmarks, is the curriculum alignment. Only six schools ranked 
themselves “Most Comprehensive” as they believed they devoted “meaningful time” to 
curriculum alignment. In the eight schools who ranked themselves “Somewhat 
Comprehensive,” the major problems included time to align curriculum and time to 
communicate with new teachers regarding curriculum alignment. There still appears to 
be lack of knowledge by staff about state standards. What may be most difficult to 
change is that some staff believe “goals are unattainable in certain subgroups.” This is 
a component that ODE will need to continue to help schools understand and work on in 
order to change this “mind set.” 

Component #5: Support of School Staff for Reform 

Even though schools did not rank themselves in this category, they were required to 
discuss and present evidence for this component. Factors that facilitated this 
component included a shared leadership and a committed staff to sustain the program. 
The Site Councils and CSR Teams worked as one leadership group, even if not 
together, they had “cross over” staff and, at the least, had CSR as an item on the Site 
Council agenda. The requirement that the school staff had to revote each year to 
continue with the CSR grant was also a factor in giving the staff a voice and giving the 
school a sense of commitment. 

Factors that hindered this component included large staff turn over which diminished the 
support previously given to the plan and reduced the number of staff who had received 
previous professional development. “Turf issues” came to play in this component as 
well; certain teams took over a program so that the plan was not driven by all the staff. 
Technical Assistance External Providers could give considerable help to schools in this 
area. Oftentimes it is difficult for staff to ask themselves how they are working together. 
Giving feedback and advice by external providers, as an outside observer, would be 
valuable. 
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Component #6: Support Provided for Staff 

Schools did not rank themselves on this component, but were required to report. 
Guaranteed time for planning and implementation was critical for success of the plan. 
Having a district that is flexible and helps the school integrate programs was key. Early 
release time on a regular basis and on-site teacher support, such as teachers on 
special assignments, were mentioned as facilitating the implementation. 

Areas of concern included the lack of: necessary materials, support staff (i.e., 
technology, librarian), time, district support, high level district administration knowledge 
of CSR, and a full-time principal (in a small school). 

Component #7: Parental and Community Involvement 

Certainly, this is the area where schools generally found themselves weakest. Eleven 
schools ranked themselves “Somewhat Comprehensive” because they did not have 
enough in place to bring parents into the overall plan. Schools were still doing the same 
“old” parent nights, often limiting them to social activities (although this is changing). 
They did not believe they have enough parents in the planning and decision making 
process. At the high school level, staff apathy to communicate with parents is apparent. 
Finally, schools did not have enough methods to communicate with non-English 
speaking parents. 

Only three schools ranked themselves “Most Comprehensive.” Factors that made a 
difference included having community partnerships, training for staff on poverty issues, 
using a full-time parent volunteer coordinator, and being able to communicate with non- 
English speaking parents. Focusing on student achievement as a part of family 
involvement was a strength in these schools. 

Obviously, this is a challenging component. A key seems to be that staff needs 
professional development on innovative ways of engaging parents and community 
members. 

Component #8: Annual Evaluation 

Eight of the 13 schools reporting (one did not self-report) ranked themselves “Most 
Comprehensive.” By the very nature of the CSR Progress Reports being presented, the 
schools had to work together to prepare the report. The reports were often shared with 
staffs, parents. Site Councils, and even school boards. As schools prepared their 
reports, they noted that more time was spent to review Action Plans and data, staff 
committees gave input, and all stakeholders were engaged. Because a number of 
documents were required in the Progress Report, the teams had to seek out information 
and discuss with staff. In the past, as mentioned above, the written progress reports did 
not engage the entire staff and/or teams. Schools seemed genuinely proud of their 
CSR Progress Reports Presentations and, thus, ranked themselves high in this 
category. 

The five schools who ranked themselves “Somewhat Comprehensive” generally did not 
have the interim assessment data (one of the report requirements) and documentation 
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in good order. Even though this was certainly not new for Round 2 schools, it continued 
to be a problem (most definitely at the high schools). Even though interim assessment 
data would normally fall under Component #4, because it was a requirement in the 
report, schools could not rank themselves higher in evaluation if part of the evaluation 
plan was not completed. Another issue included lack of structure to analyze progress 
on their own school. 

It is noted that one school had significant media attention which helped to raise the 
community awareness, at the same time it frustrated the staff as the information and 
data was superficially reported in the press. 

Component #9: External Assistance 

While schools did not rank themselves, this was an area of growth in Round 2 schools. 
By the end of three years of implementation, the schools were more “savvy” to whether 
they were getting the support they needed. If a Model Provider was consistent and 
making meaningful visits then the school felt supported. Even schools with a Model 
Provider (or at times when there wasn’t one), the schools contracted with other external 
providers who were more knowledgeable in school reform and who could step in and 
help the school in various ways (i.e., data analysis, parent/family involvement, staff 
commitment). 

The schools felt hindered in this component when the schools felt the model provider 
was lacking understanding of the schools’ needs or there was a conflict in the contract 
or staff compatibility with the Model Provider. 

Component #10: Resources 

While schools did not rank themselves, they were required to document how CSR funds 
were spent and how funds were coordinated with other resources. Factors that 
facilitated this component included a good integration of funds and other district support, 
restructured time and schedules to make the program work, lasting partnerships which 
brought in-kind funds to the school, district flexibility between programs, and efficient 
budgeting. 

Schools who felt hindered in this component remarked that lack of planning on how to 
sustain the program after CSR funds are gone was a major concern. Sometime there is 
a conflict with a change in district personnel or a change in district priorities and goals; 
budgeting becomes an issue often in these cases. 



Component #1 1 : Strategies that Support Academic Achievement 

Schools did not rank themselves in this area. In the CSR Progress Report, schools 
were required to discuss non-instructional strategies that supported student 
achievement. Some schools were able to understand this component and had plans 
such as school climate, attendance, and behavior. Schools that felt they understood 
this component had staff receive training on non-instructional strategies, and they had 
done some research on the strategies. 
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Schools who did not understand or feel they had a good plan in this component, 
generally felt they had a lack of knowledge of research on non-instructional 
components. Some schools felt that the Model Providers did not address these areas 
which meant it was often up to the schools to “figure it out.” This meant time and 
resources which schools did not have and so this component was weaker. 

Since this was a new CSR component to Round 2 schools, from the original nine CSR 
components, this may account for a lack of understanding. It will be interesting to note 
in later Rounds, those who started with all 1 1 components, if they have a better 
understanding of all the components. 

Table 17 provides the summary of the self-reporting by Round 2 schools during the 
CSR Progress Report. 

Schools Self-Ranking on the Continuum for Assessing 
the Comprehensiveness of Schooi Reform Pians 

Round 2 



Eleven CSR 

Implementation Components 


Not 

Comprehensive 


Somewhat 

Comprehensive 


Most 

Comprehensive 


Research-Based Methods 




7 


7 


Comprehensive Design 




3 


11 


Professional Development 




3 


11 


Measurable Goals and 
Benchmarks 




3 


11 


Sub category: Curriculum 
Alignment 




8 


6 


Support to School Staff * 








Support Provided for Staff* 








Parent and Community 
Involvement 




11 


3 


Annual Evaluation 




5 


8 


External Assistance * 








Resources * 








Strategies that Support 
Student Achievement* 









'Schools did not self-rank in this component; 14 schools completed this document. 



Table 17 
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Rounds 3 and 4: Schools Progress in Implementation of the Eleven Components 

Round 3 and 4 schools are in the midst of implementation and have similar strengths 
and areas of concerns. These are general summary statements, from the SRT reviews 
of the schools’ Progress Report Presentations, but not at the level of specificity 
presented for Round 2 above. 



Strengths of Round 3 and 4 Schools 

• The CSR Team seems to be good at recognizing changes that are needed and 
making adjustments. 

• Using assessments to inform instruction more effectively is clearly happening. 

• The CSR Team has noted the importance of moving forward with a stable staff. 

• Having literacy embedded throughout the building is a positive strategy. 

• Strong teacher leadership is a definite strength. The CSR plan is teacher driven 
and site-based. 

• The staff appears to have a common language which has a powerful impact on 
students. 

• An awareness of the “culture of inquiry” supports student achievement. 

• Strong attention is given not only to current achievement but to an understanding 
of the students’ journey through school. 

• There is a clear focus on academics, but also strong plans for attendance and 
student behavior. Attention to diverse populations is apparent. 

• Schools realize the CSR process is a way to strengthen the whole school 
experience for children and staff; it isn’t just a funding source for one area. 

• Strong leadership at the school has identified barriers to student learning and has 
successfully allocated and planned for critical personnel additions. 

• Schools recognize that all subgroups need support from all staff; professional 
development is helping staff with the skills they need to work with subgroups. 

• Several schools have created a CSR Notebook for each staff member with 
specific CSR information (i.e.. Action Plans, interim assessment schedules, and 
professional development calendar, and other pertinent information). 

• There appears to be a “no excuse” attitude and a firm commitment to being 
comprehensive for total school improvement. 

• Having the school act as a true learning community helps to create a strong 
vision. 

• Professional Development has included a coaching model, feedback and on-site 
support. Reflection strategies for staff are also included. 

• The district support is obvious with “data driven dialogues,” protocols, 
observations, and “walk throughs.” 

• Alternative schools face different population challenges. They aren’t making 
excuses but rather getting in there to figure out solutions. 

• In professional development, training the entire staff at one school on “learning 
about math” and not just how to teach math strategies showed an integration of 
content and a deeper understanding. 
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Areas of Concern of Round 3 and 4 Schools 

• Schools need to work on being able to articulate specific strategies so they will 
make a difference on academic achievement. 

• Because training occurred during the summer, having new learnings “revisited” in 
the fall is important. 

• Staffing and equipment have been given a great deal of funding. This can be a 
problem if the money is not providing enough professional opportunities for staff 
and support for parents. Schools need to be sure budgets reflect the needs of 
the staff. 

• Knowing CSR is the school reform plan, not just the Model Provider, is important. 

• The lack of on-going external support should be addressed. Schools need 
“outside eyes” to help with tackling the difficult or big issues. 

• With new staff changes, it may mean a different school culture will develop. It is 
always a challenge to reach unity when welcoming and including new voices and 
new perspectives. Maintaining momentum, as staff changes, is also a factor. 

• It appears that assessment is often seen by some as an event, instead of a 
strategy for school improvement. Future staff development might target staff 
“assessment literacy,” with an emphasis on local, unique, timely assessments 
that drive instructional strategies. 

• Parent involvement needs to be looked at in deeper levels than just open house, 
math nights and brochures. 

• Leadership is a key factor in success; a school needs to be careful that the 
leadership is shared by the staff and does not rest with just one person. 

• Schools believed it was significant that they had a “common language” whether 
through a Model Provider or a Locally Developed Program. However, there were 
still issues with Model Providers, as stated above in Round 2 Component #1 . 

• Working with all staff, especially those with reservations about CSR, and trying to 
understand their concerns is critical for staff buy-in. 

• Schools continue to work to discuss the “non-discussables”. 

• Schools need to find ways to engage parents in understanding the CSR plan and 
to participate in the long-range decisions. 

• Staff needs to understand the eleven CSR components. The school may have to 
work extra hard to be sure everyone is implementing and supporting the 
components. 

• Schools need to review the Continuum for Assessing the Comprehensiveness of 
School Reform Plans and reflect on what their program truly looks like so they 
can address changes during the year to make it “Most Comprehensive.” 

• The concern of “Three years isn’t enough” is duly noted! 

• Interim assessments need to be clearly developed, teachers trained in 
administering them, and used so that instruction can change to meet the needs 
of the students. 

• Paraprofessionals must get the training and support they need. 
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• Demographics of schools are changing; schools must be sure plans address the 
needs of the subgroups. 



Tables 18 provides the summary of the self-reporting by Round 3 schools during the 
CSR Progress Report Presentations. 



Schools Self-Ranking on the Continuum for Assessing 
the Comprehensiveness of Schooi Reform Pians 

Round 3 



Eleven CSR 

Implementation Components 


Not 

Comprehensive 


Somewhat 

Comprehensive 


Most 

Comprehensive 


Research-Based Methods 




1 


6 


Comprehensive Design 




1 


6 


Professional Development 






7 


Measurable Goals and 
Benchmarks 




1 


7 


Sub category: Curriculum 
Alignment 




2 


5 


Support to School Staff * 








Support Provided for Staff* 








Parent and Community 
Involvement 




4 


3 


Annual Evaluation 




1 


6 


External Assistance * 








Resources * 








Strategies that Support 
Student Achievement* 









Schools did not self-rank in this component; 7 schools completed this document. 



Table 18 
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Tables 19 provides the summary of the self-reporting by Round 4 schools during the 
CSR Progress Report Presentations. 

Schools Self-Ranking on the Continuum for Assessing 
the Comprehensiveness of Schooi Reform Pians 

Round 4 



Eleven CSR 

Implementation Components 


Not 

Comprehensive 


Somewhat 

Comprehensive 


Most 

Comprehensive 


Research-Based Methods 




1 


6 


Comprehensive Design 




2 


5 


Professionai Deveiopment 




1 


6 


Measurabie Goais and 
Benchmarks 




1 


6 


Sub category: Curricuium 
Aiignment 




4 


3 


Support to Schooi Staff * 








Support Provided for Staff* 








Parent and Community 
invoivement 




3 


4 


Annuai Evaiuation 




2 


5 


Externai Assistance * 








Resources * 








Strategies that Support 
Student Achievement* 









Schools did not self-rank in this component; 7 out of 8 schoois compieted this document. 



Tabie 19 



Summary and Findings of SEA Evaiuation 

• All CSR eleven components were implemented to varying degrees by all Round 
2, 3 and 4 schools. The schools self-ranked and/or self-reported on the 
components. Schools are moving in the right direction. 

• Schools reported that it was helpful to develop a strong implementation plan in 
the beginning and to have a clear process throughout. Regularly scheduled 
meetings, review of the action plan and data throughout the year is important. 

• Schools reported that having an on-site/school CSR facilitator was instrumental 
in keeping the plan on track. 
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• As schools presented their progress reports, they identified barriers to successful 
implementation. However, some did not address these barriers or how they 
would solve them. These issues will provide areas to discuss with Technical 
Assistance External Providers (for Rounds 3 and 4) as they work with schools. 

• All schools need intensive technical assistance in development of an assessment 
and evaluation plan. Selection of appropriate local interim assessment measures 
and general awareness of evaluation methods was a challenge for many 
schools. As indicated during the Progress Reports Presentations, some schools 
still did not have adequate structure to analyze progress. The plan for Technical 
Assistance External Provider support in 2004-2005 will help these schools 
(Round 3 and 4) and districts in order to improve this component. 

• The factors that facilitated successful implementation would seem to verify 
Oregon’s approach to a Two-Step process in selection of CSR schools. Factors 
such as staff buy-in and willingness to change, a strong principal, and support 
from the district were assessed in the pre-application phase of Oregon’s Two- 
Step Application process. Schools indicated that these were among the most 
important factors in school reform efforts at this level. 



• Parent involvement is definitely an area that staff needs professional 
development help and innovative approaches to the problems of engagement 
with families. Schools continue to struggle with this communication piece 
whether it is in language challenges, parents attending events, volunteer 
coordination, and a variety of other issues mentioned by schools. 

• Schools need time to create a Locally Developed Program, to do a thorough 
needs assessment, to identify research-based models, and to develop an 
effective plan. Lack of time was the most common factor that hindered 
successful implementation. 

• Having the Continuum for Assessing the Comprehensiveness of Schooi Reform 
Rians components match the 1 1 CSR Components will be important, not only for 
school reflection, but for ranking on the 2004-2005 Progress Report 
Presentations. 

• Some schools believe goals are unattainable for certain subgroups. This is a 
national issue because of all subgroups needing to meet benchmarks. Schools 
find this difficult and especially given only a three-year implementation cycle of 
CSR. 

• Schools feel the time pressure of only three years of funding as a factor that 
looms in front of them continually. Sustainability is certainly a question asked of 
schools throughout the entire process. Research on the length of 
implementation is interesting to note. In the “Comprehensive School Reform and 
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Student Achievement, A Meta-Analysis” published in November 2002 by 
Geoffrey Borman of the University of Wisconsin-Madison (and others), it is 
stated: “...the number of years of model implementation has very important 
implications for understanding CSR effects on student achievement. The strong 
effects of CSR beginning after the fifth year of implementation may be explained 
in two ways: a potential cumulative impact of CSR of a self-selection artifact. 
Specifically, schools may be experiencing stronger effects as they continue 
implementing the models, or it may be that the schools experiencing particular 
success continue implementing the reforms while the schools not experiencing 
as much success drop them after the first few years... These studies, therefore, 
may underestimate the true potential of CSR for affecting change in schools and 
for improving student achievement. Stronger evidence is needed to understand 
the linkages between years of implementation and school improvement and, 
ultimately, its impacts on student outcomes.” 

• CSR need to be connected to other state improvement efforts and processes. All 
state level school improvement staff members working with school reform need 
to be communicating. This would not only keep the SEA informed in working 
with schools, but would avoid overlapping and conflicting information being 
presented. 
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